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distortion between the source, X, and its representation, ( 
.X) , and I (X; ( 

.X)) is the Shannon mutual information between X and ( 
.X) . The mutual information is defined as 



...the information-theoretic statistics literature and its relationship to 
the pioneering work of C. E. Shannon was given by Soofi (1994) . 

2.2 Asymptotic Rate Distortion Theory Results 

Here we give . . . 
...the distribution of X. 

The first result suggests that any choice of the number of clusters 
based on the distortion curve or monotone transformations thereof will be 
admissible in the sense... 

...to establish. Sugar (1999) gave a proof of convexity under certain 
hierarchical restrictions on the clustering methodology. Results (II) and 
(III) follow from the maximum entropy property of the Gaussian. Versions 
of (II) exist for more-complex covariance structures. However, it... 

... of clusters . 

Most of the fundamental work in this area is due to C. E. Shannon , 
who pioneered the field of mathematical communication ( Shannon 1948) and 
introduced the notion of a rate distortion function ( Shannon 1959) . Cover 
and Thomas (1991) present a more complete development, including extensive 
references and proofs... Y. T. (1985), "A Criterion for Determining the 
Number of Clusters in a Data Set," Biometrics , 44, 23-34. 
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